NO PRIORS: ARTIFICIAL INTELLIGENCE | MACHINE LEARNING | TECHNOLOGY | STARTUPSHOSTED BYCONVICTION | POD PEOPLE
At this moment of inflection in technology, co-hosts Elad Gil and Sarah Guo talk to the world's leading AI engineers, researchers and founders about the biggest questions: How far away is AGI? What markets are at risk for disruption? How will commerce, culture, and society change? What’s happening in state-of-the-art in research? “No Priors” is your guide to the AI revolution. Email feedback to show@no-priors.com.
Sarah Guo is a startup investor and the founder of Conviction, an investment firm purpose-built to serve intelligent software, or "Software 3.0" companies. She spent nearly a decade incubating and investing at venture firm Greylock Partners.
Elad Gil is a serial entrepreneur and a startup investor. He was co-founder of Color Health, Mixer Labs (which was acquired by Twitter). He has invested in over 40 companies now worth $1B or more each, and is also author of the High Growth Handbook.
Of of sorts and, like, out comes out, like, something like Dolly 2 or Dolly 3. It turns out that it's just way more complex than that and way more complex than I even imagined. I had a sense that it was more complicated than that, but then it's still further. It's more complicated than even that. You know, so I think there were a couple things that we did. One of the things that we were really focused on with that model was that we wanted to see how far we could push the architecture of something that already existed This was mostly like a test. It was a test to see whether, how far we could get as a research team before, like, the next model change. And so we wanted to take something that we knew as a recipe that worked already, which was stable diffusion Excel's architecture, which is like a unit. Right? And clip and and the same VIE that Robin Rambach trained, all this stuff. And then we sort of said, okay. What if we try to get something that's just at least better than SD Excel, the bet better than the open source model. I mean, we weren't really sure by how much. And so our only goal was to, like, just be better try to deliver on the number one state of the art open source model that we could release. And, and so we kind of learned 2 things. One is that when we looked at some of the images from something like SDXL, we noticed that there was sort of this, like, average brightness. It was really confusing. It didn't quite have, like, the right kind of color and contrast And in fact, I became so used to this. I became so surprised about the average brightness when comparing it to the images of our model that I thought it was a bug during eval. Like, I literally was, like, looking at the images, and I was like, these cannot be the right images. And my team was sort of like, hey. I think you're actually just getting used to the images of the the new model. And so we employed this thing called, like, this EDM formulation, which, like, samples a noise slightly differently. And it's like a really clever kind of math trick and there's pro there's a paper that you could probably read on it. But it it's surprising how this, like, one little, like, very clever trick can produce images that have, like, an incredibly, great color in contrast, but, like, the blacks are really vibrant with, like, a bunch of different colors as average brightness kind
Off of AI. And so that's, you know, 6,000,000,000, 5,000,000,000 annualized, and it's probably still growing. And so if you look at it from that perspective, there's a strong incentive to fund these things because they're driving so much utilization and usage. So I I definitely think we'll see more funding going into the market. I think on the foundation model side from a venture capital or angel investor perspective, I think we're gonna see fewer new language models, but we should see models in a lot of other areas. And, you know, we have new things happening in music. We talked a little about text to speech with 11, but then there's a bunch of other areas around video, image gen, physics models, biology models, material science, robotics, etcetera. And so there's this broad swath of other types of foundation models that are starting to get funded or who are accelerating in terms of the funding cycles there. And so one can anticipate we'll see a similar thing there where we'll probably have venture capitalists, do the first set of rounds, and then it'll shift over time to large strategic players who really view that as things that are beneficial. And then there may be other areas where, you know, people are doing really interesting things. Applied intuition is a good example of a company that's doing, simulation software. And, you know, they've been doing really interesting things in terms of, like, modeling behavior there for years now. Right? So I just think there's a lot of, a lot of room to still do lots of interesting things on the foundation model side, but I I do think it's gonna continue to shift over time. What domains do you find this interesting, or what's your framework for figuring out which of these things are gonna be not only important societally, but also good businesses? One basic, way to look at this, which is what are the capabilities that, we are still missing or struggling with. Right? And so one, one thing that I've been interested in for a long time is just how do you operate on time series with, more general knowledge and reasoning. Right? There are so many ways in which
We work in the office every day, 5 to 7 days a week. I spend I spent basically the 1st year, like, hyper focused on building the team. How do we build, like, the world's best engineering team and who are those people? And I built an org chart. You know, I'm at the top, and we basically built out the teams with in detail of, like, who this all these groups should look like, whether it's, you know, controls, AI, actuation, battery systems, kinematics, integration and tasks, industrial design, all of this, and then the skill sets underneath there. So it could be motor. We have, like, a, you know, rotor, stator, transmission, sensors, thermals, motor controls. That all makes sense as, like, a picture of it. Yeah. But then, like, the reality of somebody who spends a lot of time recruiting for Okay. Of these stage companies is, like, I can't just go, like, pick up that guy up from Boston Dynamics because I I decided he's the right guy. Yeah. So I then went out and found everybody online that I thought was the best in the world, and then I did 300 phone calls over 6 months. And I cold emailed all of them. So Jerry picks up and he's like, sure, Brett. They don't say sure on the first call. I wish they did that. So a few phone calls later, yeah. A few meetings later, they do. Yes. Or a certain percentage will. Yes. This is no different than what I did at Vere. I built, you know, the the first few hundred, like I said, Archer. And, yeah, the first, you know, 30 to 50 over the 1st year, I, I identified the role in the org chart, what skills were necessary. I went out and found the right ideal, person. I cold emailed. I did phone calls. I closed them. I gave them offer letters. I wrote their 30, 60, nineties, I brought them in, I worked on them with a shared vision of what to do, and I worked with them day to day next door to them. I literally I literally sit right there on the floor with everybody else and I attend every engineering meeting and I work with them on designing the product and making those trades locally on speed and time
Types of research papers around agents that we see. We see some around, like, planning for agents. So there's a bunch of papers that do kind of like an explicit planning step upfront. And then there are, other research papers that do a bunch around reflection. So, like, after it after, an agent does something, is this actually right? How can I kind of, like, you know, improve upon that? And I think both of those are basically trying to get around the shortcomings of LLMs and that, in theory, they should do that automatically. Right? Like, you shouldn't have to ask an LLM to plan or to think about whether what it's done is correct. You should know to do that, and then it can kind of, like, run-in a cycle. But we see a lot of shortcomings there. And so I think planning the ability of LLMs is is a big one, and that will get better over time. The last one is maybe a little bit more vague, but I think even just as builders, we're still figuring out the right ways how to make all these things work. What's the right information flow between all the different nodes? In order to get those nodes, which are typically an LLM call, to work, do you want to do few shot prompting? Do you want to fine tune models? Do you want to just work on improving the instructions and the prompt? And so I think there's a lot of, how do you test those nodes? That's a big thing as well. How do you get confidence in your LLM systems and LLM agents? And so I think there's a lot of workflow around that to to kind of, like, be discovered and and figured out. One thing that's sort of come up repeatedly relate relative to agents has just been, like, memory. And so I wasn't sure how you think about memory and implementing that and what that should look like. And because it seems like there's a few different notions that people have been putting forward, and I think it's super interesting. So I was just curious about your thinking on that. I also think it's super interesting. I have a few thoughts here. So I think there's maybe 2 types of memory, and they're and they're related, but I'll draw some distinction between kind of like system level procedural memory and then like personalization type memory. So system level memory, I mean, more like, what's the right way to use a tool? What's the right way to accomplish this objective?
How well you can use the GPU, how you can scale across multiple GPUs, and how you can honestly, like, be really, really up to date with the latest things that are happening in open source and research. You know, we've partnered with a company with with NVIDIA, with a company. We've partnered with NVIDIA, and and and really, like, worked really closely with their l, LLM engine called TRTLM. And that's actually driven a lot of the performance gains that we've worked with, and, you know, we've contributed to that. We've forked that. But, you know, the the hard thing there, a lot of the optimization you're doing is pretty low level, and there's no real abstraction. So you either have to learn how to use Open Software or rewrite some of these kernels by yourself. You know, if you look at something like OpenAI, like, what do people complain about? All the time, it's speed. It's speed. And, like, that's probably, you know, one of the core performance advantages of open source is that you can get these smaller smaller, smaller models to run faster. And I think, you know, that will continue to be a massive focus for us going forward as well. On the, on the benchmarks, I think, you know, it's pretty crazy how that's evolved as well. I think, you know, we've gone from, like, state of the art being 90 tokens a second, then, you know, it it got over a 100. Now it's over 200. Now we're talking some people up to 300, 400. And I I think that's continue that's gonna continue to be, like, a very, very important place to innovate in. We we think over time, we it will get somewhat commoditized, the performance that especially especially for language models, To be honest, I think, you know, more and more of that stuff should run locally, to to some degree, I think. And but being on top of it and making sure that we're we're kind of attached to the state of the art is, you know, a if if we're not existential risk to the business, and so, you know, we
